1. Field of the Invention
The present invention relates to multimedia. More specifically, the present invention relates to creating an output media signal based upon an input media signal.
2. Background Information
The combination of audio and video has become a popular means of entertainment in film, television, video games, computers, PDAs, the internet, kiosks, karaoke, ambient environments, and live events. It is extremely desirable to correlate a moving image to a sound track, or vice versa. This technique is demonstrated in film, television, music videos, video games, web sites, cartoons, clubs, events, concerts, light shows, CD-ROMs, art installations, etc.
Current technology requires the user to manually edit a moving picture with a soundtrack, either by editing frames or altering the playback rate of the audio or video. This process requires highly skilled users to work with complex editing tools and understand music structure so as to edit in synchronicity with a sound track. This traditional editing process is both very expensive and time consuming. In fact, one of the more costly and time consuming aspects of professional production of film, video, and multimedia is synchronizing audio and video sources.
Recent developments in home consumer level systems capable of playing and manipulating video has created a demand for an editing system that creates motion pictures synchronized to an audio track. This process is too complicated for the average user to perform utilizing current technologies. Under the status quo, a user has to correlate an audio track to moving images by either:
1. Simultaneously capturing audio and motion pictures, which has significant problems; In order to have a correlation between the images and the audio, elaborate pre-production choreography is required; Another disadvantage of this simultaneous audio/video capture is that the audio and video become dependent; This restricts the user from easily changing either the audio or video of the synchronized piece; or
2. Manually editing a motion picture, cutting scenes in relation to changes in a soundtrack; This technique requires the user to detect appropriate cues in a soundtrack, such as a snare drum, and then edit the picture at that moment in time; Changes in the picture or audio portions of the program require manually re-setting these in and out points, or utilizing a semi-automated technology whereby the user has to manually correlate the sound cue to the desired frame and lock the relationship in the time line.
At live events, video is frequently used to enhance the viewer's experience. Current technology allows only basic automated visual effects in response to audio. This is demonstrated by strobing lights and lasers triggered by audio characteristics or, in the case of video, image manipulation of color, hues or similar attributes. More complex correlations between audio and motion pictures typically utilize a process in which a user pre-programs video effects, and manually triggers such effects through an input device, such as a keyboard, mouse, or trackball. The status quo lacks tools for generating real time video content derived from dynamic media signals, which do not require analysis or input from a user.
The creation of a series of still images synchronized with a soundtrack requires manually setting the image frame rate to the tempo of the audio. Changing the audio to synchronize to the picture requires the user to edit the sound or manually alter the playback rate. Audio with an inconsistent tempo requires the user to manually determine where significant audio cues reside and edit the still images to this sequence. Any changes in audio require the process to be repeated for the modified portion.
The ability to quickly, simply and iteratively produce new media content is of special interest in contexts such as movie making. In particular, home consumer production of movie content suffers from the lack of easy to use yet powerful composition tools that do not require video or audio analysis by the user. Currently there are no tools that allow the user to easily synchronize or edit a motion picture or series of still images to an unrelated audio track or tracks. In addition, there is no mechanism by which pre-existing recordings can be easily and efficiently combined to present the desired effect. This is particularly true for non-professionals.
In summary, there is a need for a time-based media processing system that is capable of providing high-quality, adaptive media productions without requiring a significant level of skill on the part of the user, and is therefore suited for use by both the professional and average consumer. There would be great utility in a tool in which two unrelated media signals can be combined to create a motion picture synchronized to a sound track or other media signals and vice versa.